Transparent and Shaped Stiffness Reflection for Telesurgery

status: publishe

ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools

Real-time tool segmentation from endoscopic videos is an essential part of many computer-assisted robotic surgical systems and of critical importance in robotic surgical data science. We propose two novel deep learning architectures for automatic segmentation of non-rigid surgical instruments. Both methods take advantage of automated deep-learning-based multi-scale feature extraction while trying to maintain an accurate segmentation quality at all resolutions. The two proposed methods encode the multi-scale constraint inside the network architecture. The first proposed architecture enforces it by cascaded aggregation of predictions and the second proposed network does it by means of a holistically-nested architecture where the loss at each scale is taken into account for the optimization process. As the proposed methods are for real-time semantic labeling, both present a reduced number of parameters. We propose the use of parametric rectified linear units for semantic labeling in these small architectures to increase the regularization ability of the design and maintain the segmentation accuracy without overfitting the training sets. We compare the proposed architectures against state-of-the-art fully convolutional networks. We validate our methods using existing benchmark datasets, including ex vivo cases with phantom tissue and different robotic surgical instruments present in the scene. Our results show a statistically significant improved Dice Similarity Coefficient over previous instrument segmentation methods. We analyze our design choices and discuss the key drivers for improving accuracy.Comment: Paper accepted at IROS 201

Deep Sequential Mosaicking of Fetoscopic Videos

Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.Comment: Accepted at MICCAI 201

Control of a hybrid robotic system for computer-assisted interventions in dynamic environments

International audiencePurpose Minimally invasive surgery is becoming the standard treatment of care for a variety of procedures. Surgeons need to display a high level of proficiency to overcome the challenges imposed by the minimal access. Especially when operating on a dynamic organ, it becomes very difficult to align instruments reliably and precisely. In this paper, a hybrid ro-botic system and a dedicated robotic control approach are proposed to assist the surgeon performing complex surgical gestures in a dynamic environment. Methods The proposed hybrid robotic system consists of a rigid robot arm on top of which a continuum robot is mounted in series. The continuum robot is locally actuated with McKibben muscles. A control scheme is adopted based on quadratic programming framework. It is shown that this framework allows enforcing a set of constraints on the pose of the tip, as well as of the instrument shaft, which is commanded to slide in and out through the entry point. Results Through simulation and experiments it is shown how the robot tool-tip is able to follow sinus-oidal trajectories of 0.37 Hz and 2 Hz, corresponding to motion due to breathing and heartbeat respectively, while maintaining the instrument shaft pivoting nicely about the entry point. The positioning and tracking accuracy of such system is shown to lie below 3mm in position and 5 • in angle. Herbert De Praetere is with UZ Leuven, Cardiac surgery, Conclusion The results suggest a good potential for applying the proposed technology to assist the surgeon during complex robot-assisted interventions. It is also illustrated that even when using flexible hence relatively safe end-effectors, it is possible to reach acceptable tracking behaviour at relatively high frequencies

High-Fidelity Low-Cost Synthetic Training Model for Fetoscopic Spina Bifida Repair

BACKGROUND: Fetoscopic Spina Bifida repair (fSB-repair) is increasingly being practiced, but limited skill acquisition poses a barrier to widespread adoption. Extensive training in relevant models, including both ex- and in-vivo models may help. To address this, a synthetic training model that is affordable, realistic and allows skill analysis would be useful.OBJECTIVE: To create a high-fidelity model for training the essential neurosurgical steps of fetoscopic spina bifida repair using synthetic materials. Additionally, we aimed to obtain a cheap and easily reproducible model.STUDY DESIGN: We developed a three-layered silicon-based model resembling the anatomical layers of a typical myelomeningocele lesion. It allows for filling the cyst with fluid and conducting a water tightness test post-repair. A compliant silicon ball mimics the uterine cavity, and is fixed to a solid 3D printed base. The fetal back with the lesion (single-use) is placed inside the uterine ball, which is reusable and repairable to allow practicing port insertion and fixation multiple times. Following cannula insertion, the uterus is insufflated, and clinical fetoscopic, robotic or prototype instruments can be used. Three skilled endoscopic surgeons each did six simulated fetoscopic repairs following the surgical steps of an open repair. The primary outcome was surgical success, based on water tightness of the repair, operation time &lt;180 minutes and an Objective-Structured-Assessment-of-Technical-Skills (OSATS)-score of ≥ 18/25. Skill retention was measured using a competence commulative sum (C-CUSUM) analysis on composite binary outcome for surgical success. Secondary outcomes were cost and fabrication time of the model.RESULTS: We made a model for simulating spina bifida repair neurosurgical steps with anatomical details, port insertion, placode release and descent, undermining of skin and muscular layer, and endoscopic suturing. The model is made with reusable 3D-printed molds with easily accessible materials. The one-time startup cost was 211€, and each single-use simulated MMC-lesion costs 9.5€ in materials and 50 min working hours. Two skilled endoscopic surgeons performed six simulated three-port fetoscopic repairs, while a third used a Da-Vinci surgical robot. Operation times decreased over 30% from the first to last trial. Six experiments per surgeon did not show an obvious OSATS-score improvement. C-CUSUM analysis confirmed competency for each surgeon.CONCLUSION: This high-fidelity low-cost spina bifida model allows simulated dissection and closure of a myelomeningocele lesion.</p